Papers with F1 measures
Inducing Stereotypical Character Roles from Plot Structure (2021.emnlp-main)
Copied to clipboard
| Challenge: | Stereotypical character roles are important aids to narrative understanding and are often referred to as archetypes or dramatis personae. |
| Approach: | They propose an unsupervised method for learning stereotypical roles given only structural plot information using Vladimir Propp’s structural theory of Russian folktales. |
| Outcome: | The proposed method induces six out of seven of Vladimir Propp’s dramatis personae with F1 measures of up to 0.70 (0.58 average), with an additional category for minor characters. |
CoPHE: A Count-Preserving Hierarchical Evaluation Metric in Large-Scale Multi-Label Text Classification (2021.emnlp-main)
Copied to clipboard
| Challenge: | Large-Scale Multi-Label Text Classification (LMTC) tasks with hierarchical label spaces include automatic assignment of ICD-9 codes to discharge summaries. |
| Approach: | They propose a set of metrics for hierarchical evaluation using the depth of the ontology to evaluate the predictions of neural LMTC models. |
| Outcome: | The proposed metrics compare with previous evaluations on prior art models for ICD-9 coding in MIMIC-III and propose further avenues of research involving the proposed representation. |
Multi-Level Structured Self-Attentions for Distantly Supervised Relation Extraction (D18-1)
Copied to clipboard
| Challenge: | Existing approaches to label large-scale data are inadequate for distantly supervised relation extraction (DS-RE). |
| Approach: | They propose a multi-level structured (2-D matrix) self-attention mechanism for DS-RE using bidirectional recurrent neural networks. |
| Outcome: | The proposed framework significantly outperforms baselines on two publicly available DS-RE datasets in terms of PR curves, P@N and F1 measures. |